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FOREWORD 



This document is Part I of a three part report on the development 
of a generic State Educational Agency Data Analysis Plan (SDAP - Model I). 
This development has been performed by Scientific Educational Systems, Inc., 
under subcontract to the Ohio State Department of Education acting as 
agent for the Joint Federal/State Task Force on Evaluation and the U. S.^ 
Office of Education. This effort was undertaken at the request of the 
User's Guide Task Force (a sub group of the Federal/State Task Force) 
interested in the development and dissemination of products usable by 
the State Educational Agencies (SEA's). This generic SDAP intended 
to be usable by SEA's as an illustrative model of how State and Federal 
data can be used to answer some of the many diverse information needs 
related to management of educational support programs operated by and 
within the SEA's. 

Part I of this report presents additional background information 
as well as describing the developmental process followed in producing 
the SDAP - Model I. Part I contains an overview of the entire effort 
and the resulting products and discusses how these products can be used 
by sea's as a model for their own development of specific data analysis 
pians. It also contains a detailed description of the Developmental 
Process which was followed in this effort. This latter product ir 
intended to serve as a general guide to SEA personnel interested in 
either further extension of the current products or undertaking the 
independent development of data analysis plans for their SEA's. 

Part II contains the SDAP - Model I, itself, the major product of 
the study. This consists of a set of analysis statements directed at 
defining answers to a ser-'es pf questions pertinent to the management of 
SEA programs. The SDAP itself is presented in a series of tables 
containing: (a) the questions derived through the study for each of 
fifteen programs found common to the two SEA's studied; (b) the suggested 
data analysis to provide the answer sought; and (c) the probable source 
of the data required for analysis, (these sources include both State and 
Federal data streams.) This Part also presents a description of what each 
of these elements is and how the plan can be utilized within the context 
of the SEA data analysis concepts. 



Part III consists of a compendium of data available by program within 
the studied SEA's, The compendium provides a direct comparison of the 
information elements which are available by program in the two SEA's and 
in the Federal /State data system, lliese comparisons form a partial 
basis for the SDAP tables, since the data availability was used as a 
direct indication of where a given type of data or information could be 
found at the SEA level. This part also contains explanations of the tabular 
presentations and of special notations used in these presentations. It 
also briefly defines the relationship of this compendium or a similar 
SEA-specific one in the utilization of the SDAP - Model I by other SEA's, 
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INTRODUCTION 

This report describes the development of a generic State Educational 
Agency Data Analysis Plan (SDAP - Model I) to serve as a model for 
similar developments by individual SEA' This Part I of the report 
includes two major sections which present, first, an overview of the 
development and of the products and their utilization and, second, a 
rather detailed description of the process employed by Scientific Educational 
Systems, Inc., (SES) in developing this plan. The actual SDAP - Model I 
and its companion Data Compendium are presented in Parts II and III of 
this report respectively. 

SECTION A - OVERVIEW OF SDAP - MODEL I 

This section briefly describes some of the background of the study 
and summarizes the concepts and purposes of the SDAP - Model I development. 
It then provides an overview of the developmental process and of the 
products of the study. It concludes with a brief discussion of the 
applications and utility of the SDAP - Model I and the associated products. 
Historical Background of the Project 

In a meeting at the Belmont House, near Baltimore, Maryland, the 
Council of Chief State School Officers and the Commissioner of Education 
of the United States agreed to the establishment of a Joint Federal/State 
Task Force on Evaluation, dated 29 August 1968. In the first of a series 
of related reports of studies of the resulting Joint Comprehensive 
Evaluation System, SES identified three broad purposes of the developing 
"Belmont" System: 

1. To provide for consolidation of the reporting requirements 
associated with Federal support of state and local educational' 
programs. 

2. To develop a system of acquiring information more useful than 
formerly for the day-to-day management of educational programs, 
and for both long and short range planning; and 

3. To provide some effective assistance to local personnel engaged 
in evaluation activities, in the form of training and report 
writing aids.* 



The Belmont Comprehensive Evaluation System. Phase I — A Review. 
Washington: Scientific Educational Systems, Inc., 22 November 1969, P. 6. 



The original Federal/State agreement clearly envisioned the development 
of a system of mutual interest, responsibility, and benefit. As a consequence, 
one of the working groups which was formed to achieve the purposes stated 
above was the User's Guide Task Force, whose basic charge was to develop 
practicable ways in which the entire enterprise might produce products 
and results which would be usable by state and local educational management 
and evaluation personnel. 

At the same time, increasing notice began to be taken of the desirability 
of planning the analysis of the various kinds of data to be produced by 
the system so that they might be most effectively applied to real management 
-and policy questions. In another report, SES noted: "However, even with 
precise individual instrument analysis plans, the necessity would remain 
for the interrelation of these plans, and for the development of a master 
plan for integrated analysis and reports generation."* Thus, in an effort 
to achieve the purposes listed above, and in response to these recommendations, 
the development of a "Master" Data Analysis Plan, covering evaluation 
reporting for Titles and target groups administered by BESE was undertaken 
by SES.*^^ 

To summarize, the Joint Federal/State Task Force was committed to an 
evaluation system of utility to the state and local management of education. 
In addition there was a specific concern with the use and application of 
information to evaluation and management questions. A User's Guide Task 
Force was set up to work with these problems. A study for the development 
of an Analysis Plan at the Federal (BESE) level was carried out (providing 
a prototype for work at the state level). These developments led naturally 
to the present study to develop a Data Analysis Plan at the state level as 
a means for improving the utility of the Federal/State evaluation data for 
state purposes. 



* Joint Federal/State Task Force on Evaluation Comprehensive Evaluation 
System— Current Status and Development Requirements. Washington: 
Scientific Educational Systems, Inc., 8 January 1970, P. 165. 

** Development of a User-Oriented BESE Data Analysis Plan. Washington: 
Scientific Educational Systems, Inc., March 1971. 



Summary of the Concept and Purposes of the Study 

The basic concept for the study was to develop a data analysis 
plan keyed to the information requirements of State Education Agency (SEA) 
managers, and utilizing data cerived from both the Federal/State System 
and from state, local and other sources. It was felt that the study 
might serve as Phase I of a longer and more specific effort, but that 
the activities of later phases should grow out of the findings of Phase I. 
Therefore, several delimitations on the study were decided upon in early 
planning sessions in order to emphasize the generalizability of the 
results and to keep the study within the bounds of available time and 
money. First, it was determined that the effort should be directed 
toward the SEA activities related to "program management/* (Initial 
discussions defined "program management** to include activities directed 
toward the management of recognized programs, and to exclude SEA internal 
management, policy planning, and legislative interaction as well as any 
other non-programmatic or non-LEA (and pupil) directed activities. Further 
definition was developed in the course of the study,) 

Secondly, the development of a model analysis plan was determined to 
be necessary because of the need for a -product more general than a plan 
developed specifically for a single SEA, or one carrying analysis down to 
the level of numerical manipulation, A model analysis plan could be 
developed based on the program management information needs of several 
sea's, rather than those of only one. And it could be constructed at a 
level of generality that would allow other SEA's to use it as the core of 
their own analysis plans, constructed on the basis of their own specific 
needs. 

Thirdly, it was decided that the model plan would be confined to 
those aspects of program management found to be common among several SEA's. 
The management of programs peculiar only to a particular SEA would not 
be considered in the model, since doing so would restrict the generalizability 
of the model. 

Finally, the development of a generic model plan, to be based on 
the commonalities of program management activities and information needs, 
obviously demanded the study of more than one SEA, But, how many? 



It was finally determined in discussions with the User's Guide Task 
Force that a detailed study of the program management activities and 
requirements in two SEA's, together with verification of their common 
elements in a third one, would be sufficient to fulfill the requirements 
for generality of the model while remaining within the time and dollar 
constraints established for the project* 

To summarize then, the User's Guide Task Force authorized the conduct 
^ of Phase I of a study to develop a State Data Analysis Plan, Phase I 
was to create a model SDAP, based upon the commonalities discovered 
among several states to ensure a certain degree of generalizability of 
the results to other SEA's, It was to concern itself only with prog ram 
management, leaving other aspects of SEA management to be dealt with in 
other efforts and later phases. It was to produce a generic model to 
serve as an illustration of the process for the use of interested SEA's, 
since the use of several states upon which to base the Plan precluded 
making the results specific to any state. Finally, the Model I, generic, 
State Data Analysis Plan for program management was intended to serve 
two broad and basic purposes defined by the User's Guide Task Force: 

1. The model SDAP should provide a direct indication of the degree 
to which Federal/State system data might be utilized in meeting 
SEA program management information requirements. This would 
result from the examination of the Federal/State system data 
sets in relation to the information needs to be specified 
during the ^study. 

2. The model SDAP should serve as an illustrative model for 
other 5EA's of how to define their own management information 
needs and analyze their own data sets to meet these needs. This 
would result from the definition of analytic procedures within 
the model to meet the common SEA information needs developed. 

Overview of the Process 

This section summarizes the developmental process employed by 
Scientific Educational Systems, Inc., to produce the generic, Model I SDAP 
for SEA program management. (The detailed process descriptions may be 
found in Section B of this report.) 
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The development of the SDAP required the inputs of many of the 
personnel of the participating SEA's, It also required the participation 
of the User's Guide Task Force and USOE personmi from time to time in 
planning sessions, SES made full u^a of the good offices of various state 
personnel in setting up an'i coordinating the visits of contractor 
personnel to the various .:ate departments during the course of the 
study. Furthermore, the inputs of these persons were regularl;- sought 
at briefings and planning meetings. 

The general procedure was based upon contractor visits to SEA 
facilities for the purposes of interviewing key program management 
personnel, of examining data streams and forms, and or reviewing descriptive 
reports with SEA personnel. The initial interviews served an orientation 
function, and were followed up with specific questions designed to 
reveal the managerial functions of the individual with respect to one 
or more defined programs of activity and the data sets or information 
needed to carry out such functions. These interviews were translated 
into Program Descriptions for each of the programs. After this procedure 
had been carried out in two states, programs common to the two states 
were identified and Common Program Descriptions developed. There were 
15 Programs found to be common to the two SEA's. The Common Program 
Descriptions were reviewed and verified in the third state, and addenda 
were prepared based upon these data. Only one common program was not 
common with the third SEA (Bilingual). All Program Descriptions and 
Common Program Descriptions were then reviewed for accuracy in Colorado 
and Connecticut, and revisions made. These then became the basis for 
the extraction of the Common Information Needs which when expanded and 
reorganized served as the requirements portion of the model SDAP. 
Finally, a compendium of data was developed from the Common Program 
Descriptions and collected data forms, and the information- requirements 
were related to generic data sources dravm from the Compendium in terms 
of the analytic procedures necessary to bring the data to bear on the 
information requirements. The construction of the Data Compendium and 
the Model SDAP were described in a Process Report (this document). 

The general procedure summarized above is synopsized in Exhibit A-1. 



Exhibit A-1 
Synopsis of SDAP Development Process 



A. Connecticut SDE Analysis 

!• Identify Programs /Persons to be examined in Connecticut. An iterative 
process, requiring conferences with the User's Guide Task Force and 
interviews with Bureau Chiefs and others, to locate personnel concerned 
with program management, evaluation, reporting and other information 
usage . 

2. Develop Program Descripti ons for each Connecticut program; consisting 
of program-related interview(s) and an analysis/synthesis of data 
derived from the interview(s) to identify: 

(a) The functions performed by the interviewee in relation to the 
program(s) in question. 

(b) The program objectives and activities and some indications about 
expected program effects. 

(c) The management decisions to be made or major questions to be 
answered for program management (by program function as possible). 

(d) The evar ation reporting requirements laid on the program manager 
and the source of requirement. 

(e) The data inputs used to meet c S d and the sources of the data 

(f) The data sets and forms which may be used by this program from 
any identifiable source. 

B. Colorado SDE Analysis 

After all Connecticut programs have been examined, proceed to Colorado; 

3. Identify Proprams /Persons to be examined in Colorado. Same process as 
Connecticut; 



Develo p ^r^ogra m Descriptions for each Colorado program. Based 
primaT ' ; / f interviews. Same process as in Connecticut. 
Extract Initial SEA Conunonalities 

5. Prepare Common Program Descriptions . Identify the common programs 
for Connecticut and Colorado and prepare Common Program Descriptions. 
Common programs based upon factors such as: 

a. Program designations (e.g., "Compensatory education services to 
the disadvantaged**). 

b. Stated program goals and objectives. 

c. Funding Sources (e.g., ESEA Title I, or similar state source). 

d. Targeted populations (e.g., directed at bilingual children). 

Verify Program Descriptions 

6. Georgia Verification of Common Program Descriptions . Common Program 
Descriptions reviewed by Georgia SDE personnel to identiJty similarities 
and differences for these Common Programs. Addenda to Com.non Program 
Descriptions prepared. , 

7. Review/Revise all Program Descriptions in Connecticut and Colorado . 
Resolve issuers raised by 6. above; check with original sources for 
accuracy; revise as neeessary. 

Final Analysis and Reporting Stage 

8. Extract Common Information Needs . Analysis of the Common Program 
Descriptions (and any other relevant data) to identify information needs 
for each program. Statement of these needs in terms of sets of manage- 
ment questions, evaluation questions, and reporting requirements. 



9, Prepare Interim Report s Describe the process of the project to this 
date; present Information Needs for each Common Program. 

10. Revise and Consolidate Information Needs . As the first step of preparing 
the Model SDAP, expand the statements and questions of Step 8, above. 
Categorize common programs into groups so that some information needs 
may be stated across programs. Develop a set of questions which is 
non-redundant (minimum set). 

11. Prepare a Data Compendium ^ Prepare an inventory of all of the data 
types which can be derived from the data sets and forms associated with 
each of the Common Programs. Display these for Colorado, Connecticut, 
and the Federal/Stave System for comparison and identification of 
overlaps in the data sets from thesa sources. 

12. Prepare a Model SDAP . Analyze the information needs in terms of the 
analytic procedures and data necessary to meet the specified needs. 
Specify data sources in each case from the Compendium. Note data sets 
unxised and data needs unmet. 

13. Prepare a Process Report . Write a narrative report detailing the 
process followed to produce the products specified in Steps 11 and 12, 
above. 




Overview of the Products 

The products of this study are: 1) the SDAP - Model I proper, 
presented in Part II of this report together with explanatory material 
directly concerned with understanding the format and content of the 
plan; 2) the Data Compendium, presented in Part III of the report, 
contains a detailed listing and comparison of the data elements found 
in the Connecticut and Colorado SEA's and in the Federal/State Data 
system; and, 3) the detailed description of the process performed in 
development of the generic d:ta analysis plan, presented in a later 
section of this Part I. 

The SDAP - Model I is the major product of this study and is intended 
to be most useful to the SEA's interested in development of their own 
information systems to support program management. The generic data 
analysis plan represented by the SDAP - Model I is based on the information 
requirements for management of fifteen programs found to be common to 
the Connecticut and Colorado SEA's. (Of these fifteen, fourteen programs 
were also found to exist in a very similar fashion in Georgia.) The 
information requirements were developed in the form of a set of management 
questions, evaluation questions and reporting questions for each program 
based on an examination of the program management activities reported 
by the SEA personnel. These program questions were expanded to some 
extent by incorporating some questions selected from sets developed by 
the New England School Development Council (NESDEC) pertinent to 
evaluation of eight of the Federally supported programs included among 
the fifteen. The resulting set of program questions comprises the basis 
for development of the actual data analytic statements for the plan. " 

As will be seen in detailed examination of the SDAP - Model I, 
these questions and analyses have been formatted in direct relation to 
the program types and programs to which they apply. The basis for this 
formatting and the way the resulting plan can be used are discussed in 
conjunction with the plan itself in Part III. 

The Data Compendium is a supportive document to the development 
and potential use of the SDAP - Model I. The Compendium contains 



a listing and comparison of all data elements available in the program 
related data sets defined by program personnel in the two SEA's studied 
in detail. It contrasts these data elements across the two SEA's 
and also compares the data with that collected by the Federal/State 
Data system. These comparisons include identification of those data 
dlements which are identical from SEA's to Federal data streams and those 
•which differ with respect to either source or level of collection, time 
of collection, or the set of category breakouts which are used. This 
product was used in the development of the analysis statements and 
identification of data sources in the SDAP - Model I, and the Compendium 
includes an identification of thpse data elements which would be 
utilized in answering program managers' questions. It is expected that 
this document, or a similar one developed specifically by a given SEA, 
would be necessary in the development of SEA specific analysis plans. 

The Process Description reported in the last section of this Part (l) 
of the report has been prepared to describe the total developmental 
process followed in preparation of the SDAP - Model I. This product 
was requested specifically by the members of the User's Guide Task 
Force after the project was already underway. It was anticipated that 
this process report would be useful to SEA personnel interested in 
undertaking such detailed analysis plans or who were currently involved 
in or contemplating development of SEA management information systems. 
It is hoped that this report can serve as a partial guide as to the 
necessary steps in development of a general analysis plan for SEA's. 
This process report is not intended to be adequate to direct the detailed 
development of such analysis plans or information systems in SEA's, rather, 
it may assist the SEA personnel in beginning such developments and may 
point out some approaches which the SEA can then either adopt or reject 
as it deems desirable. 
S EA Product Usa^e 

This project has been designed and conducted with the information 
needs of SEA program management uppermost in mind. In the original 
conceptualization, the requirement was stressed that the products should 
be usable by SEA's as illustrations of the kinds oi data analysis 
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required to meet management needs. The SDAP Model I was to be a direct 
but general level illustration of what kinds of questions need to be 
answered, what kinds of data analyses would provide answers, and where 
one might expect to find the required data elements. The User's Guide 
Task Force and the USOE believed that such an illustrative model could 
be useful to SEA's in further defining and developing their own data 
analysis and information system capabilities. The SDAP - Model I and 
the associated products presented herein have been prepared to -meet 
these purposes and to assist SEA's to meet their own needs. 

It must be remembered that the SDAP - Model I was neither conceived 
nor constructed to allow immediate implementation in any SEA, not even 
the three which contributed so much time and effort to its development. 
The basic concept of a general model to be constructed in a short 
period of time and at relatively small expense precluded the development 
of a specific level analysis plan which could be directly implemented. 
This is especially obvious when one considers the scope of the model 
SDAP - including the consideration of the information requirements of 
fifteen separate programmatic efforts at the SEA level. To develop a 
specific level analysis plan for even one large program (without the 
availability of the SDAP-Model I) would be likely to consume as much 
time and effort as has been expended in the present case. The SDAP - Model I 
was intended to be and is a model data analysis plan which should serve to 
illustrate, together with the Process Description, what the elements of 
an SEA analysis plan should be and something about how one goes about 
developing such a plan. 

The required elements of an SEA analysis plan are identical to the 
three parts of the SDAP-Model I. These are the set of questions to 
be answered to allow management functions to be accomplished; the set of 
data elements which can be or must be used to answer the defined questions; 
and the set of analysis procedures which define how the data elements 
are treated to obtain the processed information which will answer the 
questions. Each of these major elements must be defined by SEA personnel 
in development of analysis plans to support management information needs. 

11 



And, it is in the definition of these elements that the SDAP should be 
most useful to SEA personnel* First in such a development is the 
definition of the questions to be answered by the system, as stated 
above. This task is likely to be the biggest part of the development 
of an analysis plan or an informatior 3'jpport system for any purpose. 
This is because everyone in any position wants all the information he 
can get, but almost no one knows what information he really needs to 
perform his job. This means that the development of the question set 
that the analysis/ information system is to answer is both tedious and 
tough, since the analyst must tease out of the management personnel 
those questions they really need to have answered by the plan. It is 
believed that the set of management, evaluation and reporting questions 
that have been derived in this study will be directly helpful to SEA 
personnel attempting to develop such a question set. Granted that this 
set of questions is not likely to be completely exhaustive and certainly 
many of the questions asked are not as specific in either working 
or level of inquiry into the programs as some SEA personnel will want to 
make their plans; but these questions are pertinent to the information 
needs of program management at some level of specificity. Thus they may 
stimulate the thinking of both analysis and management personnel and 
thereby evoke additional or other questions relevant to the programs 
under consideration and the pecularities of that SEA. After conduct 
of the current project and the difficulty experienced in deriving even 
these geneval level questions, it is believed that any sort of starting 
point and basis for development such as this set should be most valuable 
to SEA personnel involved in such data analysis. 

Sirailai-ly, the data analysis statements presented in the SDAP • Model I 
should be helpful to persons working with such developments. Again, these 
are necessarily general in nature but they illustrate some of the basic 
techniques applicable to answering typical management questions. 

These analysis statements and the references to probable data sources 
in the SDAP - Model I should be used in conjunction with the Data Compendium 
by the SEA analyst personnel. These items should provide some meaningful 
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indications to the analyst about both the analysis processes desirable 
and some probable sources of data for the analysis. And even where the 
stated analysis is not exactly appropriate or where no similar data 
are available to the analysis, we believe that examination of these 
statements may lead the analysis personnel to solutions they might not 
have reached otherwise. As with the question set, we believe that these 
developments should stimulate thinking that might not occur without them. 
As one State person has recently put this point: "Even if everything you 
tell me turns out to be wrong, I can then tell that I shouldn't try 
that approach again." 

Although several comments above are negative (in the sense of saying 
the SDAP - Model I will not do the whole job for any SEA), it is still 
suggested that interested management personnel should carefully examine 
these documents and should ask program personnel and analysis personnel 
to do so also. Where an SEA does not already have a working information 
system, these documents could be used as a springboard to development 
of one. And even where SEA's have a working system, the questions and 
materials included here may provide the kind of stiimilation to management 
thinking mentioned above, which could result in increased information 
availability and lead to more efficient management of the educational and 
adjunctive programs in the schools. 

A brief description of how to actually use the SDAP - Model I 
format and how to extract the questions and analyses relevant to a 
single program is included in the introduction to the SDAP - Model I in 
Part II. This may be helpful also in understanding a specific usage of 
the SDAP, since it is directed toward the use of these elements as a 
model for the development of a data analysis plan specific to a single 
program* 
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SECTION B - THE DETAILED DEVELOPMENTAL PROCESS 

The basic process defined for development of a generic Model I SDAP 
was presented to an early meeting of the User's Guide Task Force and 
followed essentially the course shown in Exhibit B-1. An interim report, 
comprising the product of Step 9 on this exhibit, was submitted (October 
1971) to provide the first review of the developmental process to that 
date. The present report contains a complete and more refined view of 
the process of the entire project. 

The actual developmental process was obviously dedicated to the 
objectives of, first, identifying what program management information 
requirements were common to several SEA's, second, the identification 
of relevant data streams and forms, and third, the definition of the 
analysis processes necessary to translate data into usable information 
to meet these needs. It is also perhaps obvious that the process was 
posited on the assumption of a high degree of commonality of interests, 
activites and information needs across SEA's. This assumption was 
based on the belief that the business of education and the requirements 
of its clients in various States necessarily lead to highly similar pro- 
grammatic efforts in the SEA's to meet the observed educational and ser- 
vice needs. Thus, the developmental process was designed to capitalize 
on the underlying commonalities of the SEA's through an iterative devel- 
opment of the common areas of programs, of management activites related 
to these, and of the information needs and data availabilites necessary 
to support these. It is now believed that this assumption and the pro- 
cess have been justified by the observations made and the data collected 
in the course of the study. The actual extent of such commonalities across 
SEA's is discussed in the next section, here it can be observed to re- 
present a high proportion of the total programmatic efforts of the SEA's. 
Procedural Considerations 

The developmental process may best be described by first relating 
several procedural considerations and decisions. Then the process shown 
in Exhibit B-1 may be described in detail. 



EXHIBIT B-a 
SDAP DEVELOPMENT PROCESS 



Identification: 
Connecticut Programs/ 
Inte rvie wees 



3, Identification: 

Colorado Programs/ Interviewees 



2. Interviewing/Development 
of Program Descriptions 
RepApted Functions/Levels 
Prdkram Objectives 
Management Activities/Decisions/ 

Questions 
Evaluation Questions 
Reporting Requirements 
Data Sets/Forms Available 



4. Interviewing/ Development 
of Program Descriptions 
Reported Functions/Levels 
Program Objectives 
Management Activities/Decisions/ 

Questions 
Evaluation Questions 
Reporting Requirements 
Data Sets/Forms Available 



5. Identification of Common Programs 





! J 


6. Verification of Common Programs and 
Descriptions in Georgia 


i 





Review/ Verify/ Revise Descriptions and Data 
Sets Availab 1 e^ Col ora d o / Con nie t i eg t; 



*9. Prepare 
Interim 
Report 




3. Extract Common Information 
Needs 

Management Questions 
Evaluation Questions 
Reporting Requirements 


< 




sr 



10. Revision and Consolidation of 
Information Requirements 
Grouping of Common Programs 
Minimum Question Set 



:Federal/State 
System 
Data Sets 



*11. Preparation of Data Compendium 
Identification of State vs. 
Federal Data Set Overlaps 
Identification of Unused 

State Data Sets 



*13. Final Process 
Report 
(Phase I) 



Preparation* of Model SDAP 

Analytically Belated Infor- 
mation Needs and Dat:* Sets 
Identification of Unmet State 
Data Needs 



N.B.: Numbers are Keyed to the Text. 
* Designates Products of the Study, 
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Definition Development 

The first major problem was the need to define programs and 
program management so that these could be meaningfully used in relation 
to the SEA personnel. This problem was inherent in the original de- 
limitations of the project which confined the scope of the effort to 
"program management." This term was then ill-defined and a working 
definition had to be developed. 

The initial guidelines generated with the assistance of the 
Task Force were that internal SEA management functions, including those 
ot policy and legislative planning, were not part ojE program management; 
and, further that programs were systematic approaches to development or 
provision of educational or adjunctive services. This first cut definition 
was refined through repeated discussions among SES and SEA personnel, 
resulting in the following working definition: A program was either 
a) a systematic SEA activity which resulted in distribution of State or 
Federal funds to multiple LEA's; or b) a systematic SEA activity, directed 
toward relatively specific objectives, with a relatively immediate impact 
upon students, affecting multiple LEA's. Several rather arbitrary de- 
cisions were necessary to apply this definition during the course of the 
study, but in gen^^ral it worked quite well in identifying programs. 

An initial limitation agreed to by the Task Force was to el- 
iminate the whole area of Vocational Rehabilitation from consideration 
within this study. This decision was based in part on the nature of 
the programs within the area and in part on the opinion of the Task 
Force that most SEA's did not have jurisdiction over Rehabilitation, 
it being usually controlled by a different State agency. On the basis 
of extending this initial decision of the Task Force, SES later elimi- 
nated post-secondary programs and State-operated schools or school sys- 
tems from consideration. The former are frequently regulated by a sepa- 
rate Board (e.g. Higher Education), while the latter activities appeared 
to entail program operations to a much greater extent than program man- 
agement, per se, and also are not universally characteristic of SEA's 
(although they later appeared to be more prevalent than was initially 
believed). 
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Another definitional problem which arose was the question of 
what really constituted management of a given program and who performed 
what aspects of the management. These matters were necessarily resolved 
on an individual basis for each program through the actual interview 
process with the person(s) responsible for each identified program. The 
interview formats developed for, this purpose assisted in this resolution, 
guiding the interviews toward the functions performed by each interviewee 
with respect to various program areas. These formats were developed 
prior to visiting Connecticut and revised on the basis of initial inter- 
views there. The final form, including the list of management functions 
to which the interviewee responded, is shown as Appendix A. 
Conduct of Interviews 

Interviews were conducted with all personnel who could be iden- 
tified as connected with an SEA activity which was likely to be a program 
according to our definition. SES was greatly assisted in the definition 
of likely programs and the identification of the appropriate personnel 
for interviews by many SEA personnel in each of the states visited. 
Connecticut interviews were conducted in June and early July and Colorado 
interviews occupied late July and ea?;ly August. Interviews were conducted 
mainly on a one-to-one basis and sometimes required as much as three 
hours. (Georgia interviews for verification purposes took place in 
September. ) 

The purposes of the interviews in Connecticut and Colorado 
were: to develop an overall description of the respondent's program 
management activities and of the program requirements for other manage- 
ment, If any; to identify his loanagement information requirements re- 
lated to these activites; to identify programmatic evaluation and re- 
porting requirements; and to identify the information and data inputs, 
reports, forms, etc., related to his various program management functions. 
In this step it was found to be extremely difficult to obtain detailed 
data from some interviewees about their management activities, espe- 
cially their decision or choice functions and the information specific 
to these points. At least it was difficult to get some of the respon- 
dents to think and talk in those terms. Therefore, in some cases the 
interviews and the resulting program descriptions were less precise 
than desirable in this critical area. This problem resulted in having 
to draw inferences from the data available including the interviewer's 
notes, during the extraction process described later. 
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Program Descriptions 

The interviewer took notes and collected forms and reports for 
later study. As soon as practicable, the interviewer wrote up his notes 
in the form of a program description, the content of which is outlined 
in Exhibit It should be noted that these program descriptions re- 

present what the program manager TOLD the interviewer about the program; 
they are not the result of direct observation of the program itself. 
Thus, the validity of the program description is to a great degree de- 
pendent upon the perceptiveness of the interviewees with regard to their 
management activities. In some cases several of the interviewees proved 
to be concerned with the same program as we defined it, and in these 
cases the program description was written to reflect the views of all. 
Details of the Process 

From this point on the course of this report parallels the steps 
shown in Exhibit B-1, above. 

1* Identification of Programs/Interviewees, Connecticut 

Liaison between the contractor and SEA personnel was first ac- 
complished through the Connecticut member of the User's Guide Task Force. 
Using this person as a contact point, members of the Connecticut State 
Department of Education (SDE) were informed of the project and the im- 
pending visits of contractor personnel. On their first visit, key con- 
tractor personnel met with key SDE personnel in a series of brief 
discussions and interviews arranged by the liaison, during which the goals 
of the project were discussed and cooperation for further contacts was 
secured. During these firsc visits, information was collected regarding 
the organization of the SDE, the range of programs being dealt with, 
and the managerial responsibilities for each. In general, programs 
were eventually identified by discussing possibilities abstracted from 
written materials and prior discussions with whichever SDE personnel 
appeared cognizant. In turn, these discussions led to sharper defini- 
tions of programs, and the identification of program management personnel 
for each. 
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EXHIBIT B-2 
PROGRAM DESCRIPTION FORMAT 



A. Program Title and Authorizing Legislation 

B. Reporter 

C. Required Program Functions 
o Planning 

o Evaluation 
o Evaluation Reporting 
o Project Administration 
o Dissemination/Reporting 
o Financial Management 
o Personnel 

D. Program Requirements 

1. Purpose 

2. Object ives/Activities 

3. Target Groups 

E. Program Management 

F* Information Requirements 
G. Reporting Requirements 



2. Interviewing and Development of Program Descriptions , Connecticut, 
As the Connecticut data collection was the first in the project, 
and as the techniques used were iteratively developed during this time, 
the data collection process for Connecticut is described below in some 
detail. 

SES's approach was to employ interviews to gain data on manage- 
ment functions and informational requirements, and to identify informa- 
tional availabilities through examination of existing forms, reports, 
and instruments used by the SDE. .Overlaps and gaps in informational 
requirements and availabilities were to be determined through comparative 
analyses of these basal data. 

. This approach was more or less formalized into the following 
steps: 

a. Identification of relevant programs and individuals 
for interviewing as described in Step 1, above. 

. b. Design of interview formats and worksheets allowing 
data in all categories to be collected in one 
interview. 

c. bchedule and conduct interviews: 

1) Modify interview format as necessary. 

2) Expand or shorten schedule ns appropriate. 

d. Perform comparative analyses. 

In general, this procedure was followed successfully. The ac- 
tual outputs of Step b were two forms, one for collecting function and 
informational requirement data and the other for recording descriptive 
data about forms containing available information. The "irst iritervi^w^ 
however, indicated that one of the forms which had been designed was 
inappropriate to the task. A new design was developed. Another inter- 
view was conducted using the new interview format and still further 
modifications seemed warranted. These were made and finally the format 
which was used to the end of the process in Connecticut was achieved. 
(As mentioned earlier, this form is shown as Appendix A.) 

At the same time, it was. discovered that an effort to estab- 
lish program objectives within the SDE had produced se^s of objectives 
written by many of the personnel to be interviewed. It seemed likely 
that these could be used during the interviews to provide partial bases 
for communication about program functions and activities during the 
interview process. 
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Also during this time data already received from the SDE about existing 
forms was compiled into a list by Bureau of form numbers and titles 
showing whether the form was received or sent out by the Bureau. It was 
decided to use this material as well during the interviews. 

An interview employing the final form was conducted successfully 
as follows: 

o The interview respondent is asked for basic identification 
data. 

o The respondent is asked to identify the functions he per- 
forms and the level of performance. ( A prepared statement 
of functional definitions is presented to the respondent 
at this point. ) 

o The respondent is asked to relate the indicated functions 
to specific programs. 

o The respondent is presented a copy of his stated objectives 
and asked to modify them^ if they are not up to date. 

o The respondent is asked to review the input and output 
forms he works with and to explain which informational 
sections or elements aid him in program management and 
how. Whenever possible copies of reports and forms are 
retained by the interviewer. Due dates and legal re- 
quirements are also noted. 

o The respondent is finally asked to cite those informational 
requirements which are not being met through existing avail- 
abilities. Again, needs are recorded in terms of the ques- 
tions the respondent wishes answered. 

With thd occurrence of a successful interview some idea of the 
amount of time required for interviewing was gained. Based on this and 
the SDE personnel identifications developed earlier, the Task Force liai- 
son was contacted and requested to schedule interview appointments. At 
this time the list of people to be seen was expanded to include those 
who were identified as responsible for the processing of specific forms 
and reports but who had not been selected earlier as key program people. 

The scheduling process did not work completely smoothly. At 
first, some required persons could not be reached, which caused delays 
in the arrangement of appointments. Then it was found that some inter- 
views took longer than others which caused blank spots in the interviewing 
day. Neither of these situations endured for more than a day or two, 
for it was <?iscovered that frequently no formal appointment was required, 
that people were often free enough to be interviewed immediately upon 
request. 
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With this realization, the interviewers asked permission to make 
their own appointments with prospective program interviewees. This "freedom 
to contact" allowed them to produce a schedule and to make modifications to it 
with little effort, in the event that an interview terminated ahead of 
schedule it was frequently possible to move directly to another respondent 
with no loss of time. In this fashion, it was sometimes possible for a 
single interviewer to accomplish three and occasionally four interviews 
in a day. 

As the interviews progressed, attempts were made to write up 
narrative program reports based on the interviews which had been conducted. 
It was discovered, however, that until an entire Bureau had been canvassed 
there was no certainty that all data about a given program within the Bureau 
had been collected. This coupled with the fact that interviewers were begin- 
ning to see two and three people a day, which left little time for other 
activities, caused program write-up activity to be postponed. 

During the next two weeks, interviewing proceeded smoothly. In 
addition to personnel directly involved in program management, interviews 
with Bureau chiefs were conducted, and by the end of this period only a 
few people who had been previously unavailable remained to be seen. It 
required three days in the fifth week to interview these and a key Associate 
Commissioner and the Deputy Commissioner, thus concluding the interview 
process in Connecticut. Interviews with Bureau chiefs followed roughly the 
same format as those with program people although considerably more emphasis 
was placed on management functions and informational needs than av analysis 
of actual reports. Interviews above the Bureau level were unstructured; 
in general they began with the interviewer' ssummrrization of the activities 
in Connecticut followed by request for comments and elaborations from the 
respondent. 

Once the interviewer had collected all of the interviews relevant 
to a particular program, he attempted to integrate these data into a single 
narrative Program Description covering the topical outline shown previously 
as Exhibit B-2. The Program Description format itself was developed itera- 
tively from attempts to write up descriptions based upon the first several 
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interviews. After going through several forms, it was decided that the 
outline shown in the Exhibit was a suitable way to consolidate and present 
the information crucial to the further development of the model. The final 
Program Descriptions were intended to be the basic documents to be used in 
identifying the commonalities between Connecticut and Colorado, as well as 
the basic source for the abstraction of the evaluation reporting and 
information requirements associated with program management functions for 
each of the common programs* 

There were 20 programs identified and studied in Connecticut* 
They are listed, along with the names of the interviewees, as part of 
Appendix B to this report* The actual Program Descriptions were presented 
as Appendix A to the Interim Report (October 1971)* 

3* Identification of Programs/interviewees, Colorado 

The process f llowed here was identical to that employed in 
Connecticut* Working through the Task Force liaison person from Colorado, 
orientation and coordination visits were accomplished, followed by the 
identification of programs and their cognizant personnel, scheduling, and 
interviewing* 

^* Interviewing and Development of Program Descriptions, Colorado 
Again, the procedure employed in Connecticut was directly appli- 
cable here* However, the same form used in Connecticut was used here 
without further iterative modification* The interviews were conducted 
in a similar manner, and required about three and one-half weeks for 
completion* Following the completion of the interviews. Program Descrip- 
tions were again written, to the same format used in Connecticut* The 
20 programs identified and studied in Colorado are also listed in Appendix 
B to this report, along with the names of the interviewees for each* The 
actual Program Descriptions for these programs were presente*^ as Appendix 
B LO the Interim Report (October 1971)* 

5* Preparation of Common Program Descriptions 

The next step involved the comparison and extraction of the 
commonalities across the programs identified and described in the two 
states* This called for a review and synthesis of the data contained in 
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the Colorado and Connecticut Program Descriptions and the identification 
of common programs, their management requirements, and their information, 
evaluation, and reporting requirements. In general, this process required 
that the analyst study the Program Descriptions prepared for each of the 
two states, along with the backup forms, notes, publications, and other 
materials. Usually it was obvious as to which programs were candidates 
for the designation cf "common". In fact, it should be noted that during 
the Colorado data collection phase, the analysts looked for the existence 
of the programs found in Connecticut, Conversely, the Connecticut data 
was reexamined in areas where Colorado appeared to have a program not found 
in Connecticut. 

In general, the analyst regarded programs as common if they served 
similar objectives in similar ways. Often common programs had the same or 
highly similar funding sources. Often' they had the s?..Tie or similar reporting 
requirements and management requirements. It was, however, a judgmental 
process. And this was particularly true in the cases where several programs 
in one statcj seemed to be subsumed under one in the second state. In such 
instances the analyst tended to define the common program as including the 
several programs in the first state. 

Once the analyst had determined that a common program existed, 
he prepared a Common Program Description, following the same format and 
outline as that used for the original Program Descriptions. The Common 
Program Descriptions emphasized the common characteristics of the programs 
in the two states, but where there appeared to be significant differences, 
these were noted in asides within the Descriptions. These Common Program 
Descriptions formed the basis for the following verification steps, and for 
the subsequent extraction of Common Program Evaluation Questions, Common 
Program Management Questions, and Common Program Reporting Requirements. 
(These three sets of questions were deemed to cover "common information 
needs", and were reported by program as products in the Interim Report 
(October 1971)). A list of the Common Program Descriptions is show in 
Exhibit B-3; the Descriptions themselves comprised Appendix C of the 
Interim Report. 
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EXHIBIT B-3 
COMMON PROGRAM DESCRIPTIONS 



1. Bilingual Education Program 

2. NDEA Title IIIA Program 

3. Migrant Program 

4. Management of General State Grants Program 

5. School Assistance in Federally Affected Areas 

6. Transportation Granf \dmini strati on Program 

7. School Library Program 

8. Dr iver Education Program 

9. Vocational Education Services Program 

10. Drug Abuse Program 

11. Special Education Program 

12i ESEA Title III Grant Management Program 

13. Compensatory Education Program 

14. School Food Services Program 

15. Adult Basic Education Program 
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6# Verification of Program Descriptions, Georgia 

In order to verify the commonalities identified between Colorado 
and Connecticut, the Common Program Descriptions were taken to the Georgia 
SEA for review. In this case the procedure differed somewhat from that 
employed in Colorado and Connecticut. There was no effort to identify 
all of the programs extant in the Georgia SDE. The data collection was 
concerned only with those programs which had been found to be common 
between Colorado and Connecticut, Furthermore, the interviews were 
confined primarily to members of the Division of Research, Planning, 
and Evaluation, since this Division had recently performed an internal 
review of programs and felt that the purposes of the project would be 
adequately served by this procedure, (The User's Guide Task Force approved 
this change in method,) These verification visits were carried out in 
September, 

The procedure used in the Georgia interviews was as follows. The 
Task Force liaison with the Georgia SDE made the preliminary arrangements 
for the contractor visits to the SEA, and coordinated the identification 
of appropriate interviewees and the scheduling of the interviews. Again 
the interviews were on a one-to-one level, though the interviewee might 
have been the respondent for more than one of the programs. The procedure 
used was to ask the respondent first to summarize briefly the salient details 
of the program in question, and then to review the Common Program Description 
for the other two states for that program. He was then asked to comment 
on the similarities and differences between the Common Program Description 
and the conduct of that program by the Georgia SDE. This method worked 
very well, and produced a number of points of commonality. The Common 
Program Descriptions were not actually revised on the basis of these data, 
but addenda to the Common Program Descriptions were prepared summarizing 
the additional data gathered in the Georgia interviews so that they might 
be included in subsequent steps. Only one of the 16 Common Programs was 
not found in Georgia — the Bilingual Program, 

7. Verification of Program Description Accuracy 

As a final verification step, the completed Program Descriptions 
and Common Program Descriptions were returned to Connecticut and Colorado 
for review by those persons who had originally served as interviewees. First 
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the completed Descriptions were mailed to the interviewees for their 
review, and then visits were made to these two SEA's to pick up the 
reviews and discuss them. This step was taken in order to resolve a 
few issues which had arisen during the course of the preparation of the 
Descriptions and/ or the collection of the verification data in Georgia. 
In addition, it was felt that such a step would reduce the possibilities 
of transcription and interpretive errors which may have otherwise crept 
into the final versions of the Descriptions. Comments and critiques 
were usually brief and of a minor nature. These final verifications were 
carried out during October, and appropriate revisions were made. 
8. Extraction of SEA Common Information Needs 

Common information requirements for each Common Program were 
derived from the available data about the program management, evaluation, 
ar.d reporting activities as reported in the course of data collection 
described above. These common requirements were developed within three 
areas: management questions; evaluation questions; and reporting 
requirements. After additional analysis and revision (below), these 
information requirements constituted the basis for the definition of the 
analysis procedures of the Model SDAP. 

The first set of information requirements for each program as 
derived through an analysis process directed toward the development of a 
comprehensive list of the types of management, or evaluation, or reporting 
information needs the program manager would be likely to have. The 
starting point for this derivation was the management description and 
information reported by program managers and all other available data 
about the programs in each SEA. It was mentioned earlier that program 
managers frequently had difficulty reporting specific management decision 
functions or talking about specific information needs to support their 
decisions. This meant that for many of the programs, specific management 
information needs were not spelled out as completely as desirable. This 
was sometimes true with respect to a program's evaluation information needs, 
also. It did appear that the data collected and the descriptions that 
had been developed included these requirements as implications rather 
than as explicit statements. Thus, the task at hand was to explicate 
these requirements based on the program managers* reports and data. 
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This task was approached through direct examination of the 
available data about each specific program and the program management, 
evaluation and reporting functions spelled out by the reporter. These 
were the basis for the extraction of the three kinds of needs: management 
questions; evaluation questions; and reporting requirements. The extraction 
process was inferential. The analysts assumed the role of newly assigned 
managers of the subject program and examined all materials and data' 
with the view of defining the questions necessary to allow the performance 
of the program functions as reported. The questions to be derived were 
to cover all aspects of information which would be considered essential 
to th6 conduct of the program. This resulted in the development of three 
sets of questions and requirements: 

1) Management questions - a set of questions asking for 
information necessary to an understanding of the program 
and essential to its operation as described; this is the 
set of information considered necessary to allow the 
manager to plan and conduct his program in the State. 
Such information includes basic information about SEA 
policies and plans which might affect the program as well 
as legislative and other guidelines about the specific 
operations of the programs. However, the major portion 

of such information needs are for data related directly to 
the choice of alternative actions at management decision 
points. These management questions include items directed 
toward both of these general information needs types* 

2) Evaluation questions - These are questions which call for 
information necessary to the assessment and appraisal of 
both the management and the effectiveness of the program. 
They relate to overall program management, and specifically 
to the decision functions therein. They seek information 
which provides the basis for determining whether or not 
the program is meeting its objectives or goals, and whether 
or not the program should continue to operate as is, or 
should be changed in some ways, or should cease to exist. 
These questions also frequently call for data which are 



requested for informational purposes as well as for 
immediate applicability to specific program management 
decisions. Such informational purposes may be served by 
data relating to specific pupil or project progress, etc., 
and frequently the same data may serve both managerial 
and informational purposes, 
3) Reporting requirements - For the first set of requirements 
no attempt was made to expand the requirements as reported 
by the SEA personnel. These were expanded later into 
questions to be answered through the SDAP development. 
This analysis was specifically pointed at the extraction from 
the program reports of all possible, meaningful, questions pertaining to 
these three requirements areas. However, there was also a conscious 
effort to avoid creation of questions which had no meaning within the 
context of the specific program under consideration. For example, the 
SEA manager has no basis for asking questions about the impact of SAFA 
funding on specific pupils, since there is no SEA control nor LEA 
accountability of these funds. 

The first set of Common Information needs derived in this manner 
was reported in the Interim Report (October 1971). 
9. Preparation of the Interim Report 

The Interim Report was prepared and submitted to the Ohio SDE 
and to the User's Guide Task Force to relate the developments of the 
project up to that time, to pres-.ut the Program and Common Program 
Descriptions, and to present the sets of management questions, evaluation 
questions, and reporting requirements identified in the first analysis 
of the data. The report was presented to a meeting of the User's Guide 
Task Force at the Burlington Hotel, .Washington, D.C,, on 2 November 1971, 
and was well received. In the process of developing the required 
questions, as described above, some inferences were made about the type 
of management required by the various Common Programs. A simple classifi- 
cation system was set up on the basis of the man^ement functions required, 
and programs were sorted into four types as defined in the next section. 
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10. Revision and Consolidation of Information Re<^uirements 

Although the Information Needs reported in the Interim Report 
included all of the information which could be derived from the data on 
hand, it was decided that some additional analysis of these information 
requirements was desirable in terms of some expansion and reorganization 
of the questions developed. Specifically, the information requirements 
for each Common Program were expanded where necessary, and the Common 
Programs themselves were grouped in an effort to identify common elements 
across the Common Programs. The intention of this effort was to develop 
a "minimum set" of the questions, thus avoiding duplication and redundancy 
in the final products. A four-stage procedure was followed to achieve 
this result: 

a. The Common Programs were examined for similarity of functions, 
objectives and/or activities and categorized on the basis 

of such similarities; 

b. The management evaluation questions were examined as to 
the needs for expansion or more specification; also, the 
reporting requirements were expanded into questions being 
asked within each report; 

c. When an expanded question set had been formulated for each 
program, these were examined across programs, within the 
categories develDped as the first step, to identify: 

1) Questions which were identical across programs 

2) Questions which were essentially identical across 
programs except for differences in target groups, 
or probable data sources, or other minor points 

3) Questions which were unique to the program for 
which developed. 

d. The last step in this program was the examination of the 
questions in subsets I) and 2) above across the program 
categories to identify any further commonality of questions. 

When these comparisons had been accomplished, the resulting 
minimum set of questions for each program or program type constituted the 
final set for which analysis procedures were developed. A description of 
the process of carrying out these steps follows. 
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The first basis chosen for grouping programs was similarities 
in program management functions. It was observed early in the data 
collection phase, that certain basic funds distribution programs were 
operated with very little opportunity for discretionary management by 
the SEA personnel. Mainly the manager performed the steps necessary to 
insure compliance with the legal basis for the funds distribution program. 
Other programs were observed to require a good bit of interactive management 
between the SEA personnel and the LEA's or other program recipients. It 
appeared that this dimension, the degree of SEA manager's control or 
options for management, might be a meaningful basis for a first categorization 
of common programs. 

Accordingly, a set of the management functions which were 
performed by SEA personnel in relation to all programs was derived and 
used as the basis for development of categories of Common Programs. 
Exhibit B-4 shows the derived list, which, though not necessarily 
exhaustive of management functions, contains the functional elements 
essential to the categorization process employed. This list of functions 
was generated directly from the reported management functions and activities 
of the program managers as indicated in the Common Program Descriptions, 

Exhibit B-5 presents the Common Programs listed within the four 
categories of program management activities which were derived on the 
basis of the differences in functions across the programs. These 
differences in program" functions from category to category are indicated 
directly by the differences in the numbers following the program name, 
where the numbers refer to the listed program functions in Exhibit B-4 
and indicate that the listed function is performed in management of this 
program. The four derived categories are described below. 

Aa - Compliance Management of Funds Distribution, t,±th lAtt^f- 

SEA Option ; This appears to be the least complex set of programs 
from the point of view of management functions performed. Management 
functions are straightforward and are related to the disbursement of 
Federal or State funds to LEA's on the basis of one or more specific 
legislative decrees. Funds are disbursed on the basis of a fixed formula, 
defined by the authorizing legislation or by highly specific guidelines 
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EXHIBIT B-4 
SEA PROGRAM MANAGEMENT FUNCTIONS 



Funds are given to LEAs (G = grant; R = reimbursement). 

Funds are allotted by formula with SEA making no decision on quality 
of proposal or how much money a particular LEA should receive, 

SEA reviews proposals, makes recommendations to Federal Government 
but has no control over which projects are funded and how much money 
a project should receive, 

SEA reviews proposals and based on the quality and content of the 
proposal, the SEA may reject the proposal or fund the project. The 
amount of funding depends on the SEA review. 

SEA consults with the LEA to help establish eligibility. 

SEA reviews grant applications for correctness, completeness, 
adherence to guidelines, etc. 

SEA consults with LEA in project planning and development and assists 
in preparation of proposals. 

SEA may assist the LEA in filling out the application so that the 
LEA will receive its fare share, of the available funds. 

SEA monitors local projects. 

SEA conducts evaluations of projects. 

SEA makes reports on expenditures, activities, and evaluations to 
Federal Government. 

SEA formulates a State Plan and/or defines statewide program objectives 
and priorities. 

SEA encourages consideration of new projects. 

SEA conducts workshops and training programs to train LEA personnel. 
SEA aids the LEAs in obtaining Federal or other outside funding. 
SEA is involved in certification of teachers. 
SEA provides technical assistance in solving project problems. 
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EXHIBIT B-5 
CATEGORIZATION OF PROGRAMS 



3 



Aa - Compliance Management, No State Options 
Transportation Grants 
School Food Services 
SAFA 

State Grants 



1r. 2, 6, 16 

1^, 2, 5, 6, 9, 10, 12, 16 



^G' 



2, 5, 6, 7b, 10 



Ig. 2, 6 



Ab - Compliance Management, State Options 
ESEA II . 
Connecticut Driver Education : 
Compensatory Education : 
Migrant . 



1^, 2, 6, 10, 11, 16 

Iq> 2, 6, 7a, 10, 11, 16 

1^, 2, 6, 7a, 9, 10, 11, 16 

1^, 2, 6, 7a, 9, 10, 11, 16 



B - Discretionary Management 
ESEA III 

Connecticut Drug Abuse Education 
Special Education 
Adult Basic Education 
NDEA IIIA 

Vocational Education 



1^, 4, 7a, 8, 9, 10, 11, 16 

1^, 4, 7a, 8, 9, 10, 11, 13, 16 

1^, 4, 7a, 8, 9, 10, 11, 13, 16 

^, 7a, 9, 10, 11, 13, 16 

Ij^, 4, 7a, 9, 10, 11, 16 

Ig, 4, 7a, 8, 9, 10, 11, 16 



C - Consulting, No Funds Distribution Activities to LEAs 

Colorado Drug Abuse Education : 7a, 9, 10, 13, 14, 16 

Colorado Driver Education : 7a, 9, 11, 13, 15, 16 

Bilingual . 3, 7a, 9, 13, 14, 16 
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interpreting the legislation, which are not modifiable by the SEA manager. 
The SEA has no discretion as to the amount of funds to be given to an LEA, 
nor whether it can receive the funds, nor as to what the LEA can do with 
the funds. Program objectives, goals, priorities and disbursement rules 
are fixed by the authorizing legislation or the guidelines. 

- Compliance Management of Funds Pis tribution^ with SEA Options ; 
This program category is highly similar to the first set and is differentiated 
only by th2 fact that the SEA personnel have some discretion as to what 
can be done with the funding. This discration is normally exercised 
through the prf:paration of a State Plan for the funds distribution program. 
This State Plan allows the SEA personnel to define and influence the 
thrust or direction of the program funds usage within zhe State. There 
is still no discretion nor authority to determine whether or how much 
money is going to which LEA's. The SEA can delay or withhold funding to 
assure LEA compliance with the State Plan. 

B - Discretionary Management of Funds Distribution ; This is 
perhaps the most complex of the program groups on the basis of the management 
opportunities and options for the SEA personnel. SEA managers here have 
optional control over whether, and how much, and for what purposes funds 
will be authorized to an LEA. The SEA personnel ha^-e relatively complete 
discretion as to which of multiple proposals prepared by LEA's will be 
funded and most frequently at what levels they may be funded (within some 
limits imposed by legislation). Discretion is exercised on the basis of 
the merits of the propos^id LEA projects and their relations to the SEA- 
prepared Scate Plan objectives, goals and directions for the State. 
Through these plans and selection of proposals to be funded, the SEA 
personnel have a great effect on the development and expansion of these 
fT'^rams in the State. 

£ - Consulting and Tec.mical Assistance, No Funds Distribution : 
These programs do not relate directly to the other categories in terms 
of complexity of management, since they a. not complicated by tne financial 
management aspects inherent in the other program types. In this type of 
program, the SEA plays primarily a consulting role uf providing technical 
assistance and advice to LEA's on project development and problems, and 
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the preparation of proposals. The SEA may also carry out certain activities 
of its own. These may take the form of worksliops, training programs and 
evaluations* 

The classification of the Common Programs by State into one of 
the above four categories is summarized in Exhibit B-6. In Connecticut, 
four prograu.s fell into Category Aa (Compliance, No State Options); four 
programs fell into Category Ab (CompM.ance, State Options); six programs 
fell into Category B (Discretionary); and one program was assigned to 
Category C ( Consulting). In Colorado, fo. programs fell into Category Aa; 
three programs fell into Category Ab; five programs fell into Category B; 
and three programs fell into Category C. 

There are two programs which were assigned to different categories 
in Connecticut and Colorado. These are Driver Education, and Drug Abuse 
Education. In Connecticut, funds for the Driver Education and Drug Abuse 
Education Programs are distributed to the LEA's on a compliance and discretic 
ary basis, respectively. In Colorado,, however, in both programs the 
funds are used at the SEA level only and there is no distribution of State 
or Federal funds to the LEA's. In both these cases, the SEA performs only 
the consulting and assistance functions with respect to LE.^ projects. For 
the other 12 programs which existed both in Connecticut and Colorado, the 
program functions were essentially the same in Doth States. This exhibit 
also shows the initial classification of the Georgia programs. It can 
be seen that four programs fell into the Aa group and three into the Ab 
class, and the remaining seven programs were classified as B-type programs. 

These program groups were then examined as to the degree to 
which the several programs within a type had identical information 
requirements. This examination led to a minimum set of questions for each 
group which satisfied a large part of the information needs of most of 
the programs within a group. 

It was discovered that some management questions could be stated 
across program categories, and at least across programs within categories; 
evaluation questions could be stated across some or all programs within 
categories; and reporting requirements were relatively specific to programs. 
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EXHIBIT B-6 
CLASSIFICATION OF PROGRAMS BY STATE 



Pro^ 


ram Nani6 


Conn. 


Colo. 


Ga. 


1 

i . 


Transportation Grants 


Aa 


Aa 


Aa 


I* 


School Food Services 


•Aa 


Aa 


Aa 




C A "PA 


Aa 


Aa 


Aa 


A 


bcace brants 


Aa 


Aa 


Aa 


c 

D . 


T70T7 A T T 


Ab 


Ab 


Ab 


/: 
0. 


Driver Education 


Ab 


C 


B 


7 . 


Compensatory Education 


Ab 


Ab 


Ab 


8. 


Migrant 


Ab 


Ab 


Ab 


9. 


ESEA III 


B 


B 


B 


10. 


NDEA IIIA 


B 


B 


B 


11. 


Drug Abuse Education 


B 


C 


B 


12. 


Special Education 


B 


B 


B 


13. 


Adult Basic Education 


B 


B 


B 


14. 


Vocational Education 


B 


B 


B 


15. 


Bilingual 


C 


C 


* 



Legend : 

Aa = Compliance Management, No State Option 
Ab = Compliance Management, State Option 
B = Discretionary Management 

C = Consulting, No Funds Distribution Activities to LEA's 
* Ga. has no bilingual program 
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These minimum sets of information requirements thus resulted in the 
development of an overall SEA Data Analysis Plan which is much more 
efficient than otherwise. However, they are not a part of this document, 
but are found in Part II, the Model I SDAP, itself. 

11- Preparation of Data Availability Compeildiutn 

It was desired to study the data availability for each of the 
Common Programs in terms of three aspects: a) The overlap in data coverage 
between the states and between the states and the Fede^^al/S tate System; 
b) The state data sets not used in program management; and c) the 
expressed state program management information needs not met by the existing 
data sets. In order to accomplish this, it was decided that a complete 
data compendium would be necessary. The Data Compendium (which forms 
Part III of the report of this project) was developed as follows; 

First the analyst assembled all of the data, including all of 
the forms, relevant to a Common Program for each of the two states, 
Connecticut and Colorado. These data were cross-checked with the Common 
Program Description to ensure that all data sources for the Program had 
been indentified. Then the contents of each of the forms were inventoried 
in terms of data types in a format showing the availability of the data 
type on Connecticut forms, Colorado forms, and Federal/State forms, by 
form and i^em number. In addition, the analyst indicated identicality or 
the source of difference where the data type was treated differently on 
one or more of the three possible sources. This procedure was carried 
out for each form relevant to the Common Program,' and then for all of 
the Common Programs. 

Thus, the Data Compendium makes clear at a glance the extent 
of data overlap between the states, and between the states and the Federal/ 
State System forms. The state data sets used in program management as 
reviewed in this project were noted in the development of the Model SDAP, 
They are indicated in the Data Compendium by asterisks. (Conversely, the 
"^"s^^ set has no asterisks.) State program management data needs currently 
unmet are included with the model SDAP itself, as indicated by those questions 
for which no relevant data to be analyzed was found available in the Data 
Compendium (blanks in the Data Source Column of the SDAP). 
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The Data Compendium has been prepared as a separately bound 
Part (III) of the products of the present project. 
12 • Preparation of the Model SDAP 

The final stage of the Phase I project was the preparation of 
the generic Model I SDAP, itself. This consisted of a number of steps, 
the first of which was to decide on a format for the Plan. It appeared 
obvious that the SDAP should be organized around the information 
requirements. In this case that meant u:jing either program groupings or 
type-of-question groupings to guide the layout of the Plan (the use of 
a matrix format employing both was ruled out because of the length of 
some of the sections of the Plan). After some deliberation, it was 
decided to subordinate type-of-question to program groupings for the 
organization of the final model SDAP. Thus, in general, the format 
begins with questions which cut across all programs, proceeds to groups 
of programs, and finally to individual programs. Within each level 
questions are grouped according to type: management, evaluation, or 
reporting requirement. Of course not all combinations of program groups 
and question types occur, but in general this is the schema for the final - 
model SDAP, which has been prepared as a separate Part (ll) of the 
products of this project. 

The second step in the preparation of the model SDAP involved 
the derivation of appropriate analytic techniques for dealing with each 
of the questions. This step was carried out without regard for the data 
or its availability, in order that the information requirements and their 
analyses might stand alone. The actual development of analysis procedures 
was straightforward and consisted of the following procedure. First, each 
of the final questions resulting above was examined to define the precise 
kind and extent of data required to provide the information requested. 
Next the data treatment which must be performed to create the information 
requested was specified. This data treatment is of course the specific 
analysis procedure required for the question. It may consist of simply 
tabulating some data from LEA's, or of summing and averaging head counts 
of pupils, or it may consist of the combination of two or more data types 
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in various ways. Some questions required the rather complex analysis 
of some special data types, but this was not required often. An example 
of a data question which might require a more complex analysis would be 
one which requested comparative data on achievement of groups of students 
within different subgroups of the student population. Such a request 
might lead to definition of a variance analysis format for purposes of 
means comparisons among groups- 

The definition of the data treatment required to answer a 
question usually implicitly defines the typical output which can be 
expected from the analysis. Where this was not so, an indication of the 
probable output of the analysis was stated. 

The final step in the preparation of the model SDAP consisted 
of indicating the available data sources for each of the analysis-question 
combinations. This step was accomplished by referring to the Data 
Compendium for the applicable Common Prograru and indicating the relevant 
data source for the data specified by the analysis to answer the question 
posed. It should be noted that the data sources had to be listed in 
generic form, although the Data Compendium was more specific, since the 
model SDAP was required to apply to multiple SEA's. Thus it was decided 
to list the data sources in terms of types of reports and forms rather 
than using the specific item numbers or form numbers for Colorado and 
Connecticut. During the process of relating the data sources to the 
analysis-question combinations, the analysts marked the Data Compendium 
with asterisks in all cases where the available data were required by 
any program question. Thus, those entries in the Data Compendium which 
do not have an asterisk represent data not used in any of the analysis- 
question combinations. The model SDAP itself indicates by blanks in 
the Data Source column where needed data is not collected by the current 
system. 

13. Preparation of Process Description 

Early in the progress of the project members of the User's 
Guide Task Force requested that the contractor undertake the preparation 
of a report which would detail the procedures followed during the development 
of the generic Model I SDAP. It was hoped that by detailing the procedures 
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followed, and mentioning some of the problems and errors encountered, 
the process would be explicated sufficiently to be of use to those SEA* s 
who might wish to develop their own models or apply this on^, as a guide 
to their efforts. In preparing this document, which is a Part (l) of 
the products of the project, SES decided to divide it into two sections. 
The first section would serve as an overview of the project, while the 
second would fulfill the requirements of the detailed process report. 
A Final Word 

The products of this project to develop a generic model State Data 
Analysis Plan for program management are comprised of three parts: 
1) The present document, containing an overview of the project and a 
detailed description of the process; 2) The Model I SDAP itself, with 
an introduction telling how it may be used; and 3) The Data Compendium, 
presenting a complete review and comparison of commdn-program-related 
data types for Colorado, Connecticut, and the Federal/State System. All 
three of these products were produced and bound separately. 

Scientific Educational Systems feels that the project has been 
successful in demonstrating the feasibility of the SDAP concept. It 
now remains to apply the model to specific programs in specific states, 
and, on the other hand, to extend the model to cover other aspects 
of SEA management and their required data support systems. Both such 
extensions are feasible and potentially highly useful to state-level 
educational managers and evaluators. Both may now be undertaken. 
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APPENDIX A 

SEA Program Management Interview Schedule 



SEA PROGRAM MANAGEMENT INTERVIEW SCHEDULE 



SES 
June 1971 



NAME^ 
DIV 



RM 



PH 



BUREAU 



MAJOR RESPONSIBILITY^ 
How long on job? 



In SDE? 



1, Which of the following functions do you perform and at which level? 

LEVELS 

Functions Program Bureau SEA 

Planning 

Evaluation 

Evaluation Reporting 

Project Administration 

Development 

Review 

Monitoring 

Report ing 



Dissemination/ 
Reporting 

Financial Management 
Personnel 



2. If you perform any of these for a specific program(s) for which one(s)? 
(Identify name in relation to summary list. If not there, find out if 
new program.) 
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For this program (s) we have a list(s) of program objectives and activi- 
ties. Please examine this sheet and tell us about any difference/errors 
or changes. (Give 2nd copy of program form to interviewee and record 
changes below. Repeat for each program indicated by respondent, 
clearly identify irtiich program): 
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PROGRAM REPORT DATA 



STATE 



NAME 



BUREAU 



DATA NAMF 




LEG. REQ./DUE DATE 
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APPENDIX B 

Program Descriptions and Reporters, by State 



ERIC 



CONNECTICUT 



Program 

Migrant Program 

State Aid Grant Program 



Education Professions 
Development Act Program 



Bilingual Education 
Program 



School Lunch Program 



Adult Basic Education 
and Adult Education 
Program 



Education of the 
Handicapped Program 



Reporters 

Alexander J. Plante, Bureau of Compensatory 
and Community Educational Services, Bureau 
Chief 

Frank Livak, Bureau of Educational Management 
and Finance, Consultant for Grants Management 

James Naughton, Bureau of Educational Manage- 
ment and Finance, Consultant for School 
Business Administration (resigned) 

Alfred Villa, Bureau of Educational Manage- 
ment and Finance, Chief 

Peter LoPresti, Bureau of Teacher Preparation 
and Certification, Chief 

Anthony Tedeschi, Bureau of Teacher Preparation 
and Certification, Consultant for EPDA 

Alexander Plant , Bureau of Compensatory and 
Community Educa^ ional Services, Bureau Chief 

John Harrington, Bureau of CCES, Consultant - 
Bilingual Education 

Kenneth Lester, Bureau of Elementary and 
Secondary Education, Consultant - Foreign 
Language 

Edith Blakeley, Bureau of Federal, State and 
Local Relations, Consultant - School Lunch 
Program 

James Dorsey, Bureau of Compensatory ;md 
Community Education Services, Consultant 
Adult Education 

Alan Hugg, Bureau of Compensatory and Community 
Education Services, Consultant Adult Education 

David Murphy, Bureau of Pupil Personnel and 
Special Education Services, Consultant ESEA VI 
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CONNECTICUT (Cont.) 



Regional Special Education 
Services Program 



NDEA Title IIIA Program 



State Supported Schools 
for the Handicapped Program 



Drug Education Program 



Compensatory Program 



Industrial Arts 
Education Program 



School Construction 
Services Program 

Improving Teachers of the 
Handicapped Program 



Driver and Safety 
Education Program 



Vocational Agriculture 
Program 

ESEA Title III Grant 
Management Program 



Vocational Education 
Services Program 



Robert Margolin, Bureau of Pupil Personnel 
and Special Education Services, Consultant, 
Grants to LEAs for Regional Special 
Education Services 

Leonard Garber, Bureau of Elementary and 
Secondary Education 

Lucien Zadro»ny ^ Bureau of Pupil Personnel 
and Special Education Services, Consultant, 
Title I, State Supported Schools 

Donald Anneser, Bureau of Elementary and 
Secondary Education, Consultant, Drug 
Education 

Thomas Crane, Consultant, Bureau of Compensa- 
tory and Community Educational Services, 
Division of Instructional Services 

Wesley Ketcham, Bureau of Elementary and 
Secondary Education, Consultant Industrial 
Arts Education 

Richard Howland, Bureau of School Buildings, 
Bureau Chief 

Joe R. Gordon, Bureau of Public Personnel 
and Special Education Services, Assistant 
Bureau Chief 

David Jacobson, Bureau of Elementary and 
Secondary Education, Consultant - Driver and 
Safety Education 

Lewellyn Turner, Bureau of Vocational 
Services, Consultant for Vocational Education 

Roger Richards, Bureau for Elementary and 
Secondary Education, Consultant for Title 
III Coordination 

Errol Terrel, Bureau of Vocational Services, 
Chief 



Robert Bennett, Bureau of Vocational Services 
Consultant for Program Development 

Richard Wilson, Bureau of Vocational Services 
Consultant for Resear;h Planning 
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CONNECTICUT (Cont.) 



State and Federal Aid 
for Library Education 
Materials Program 



Ralph Goodrich - Bureau of Elementary and 
Secondary Education, Consultant, ESEA Title II 



Rheta Clark, Bureau of Elementary and 
Secondary Education - Consultant for School 
Library Services 

John Crawford - Bureau of Elementary and 
Secondary Education - Consultant for School 
Library Services 
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COLORADO 



Program 

NDEA Title IIIA Program 

Migrant Program 

Bilingual Education 
Program 

Education Professions 
Development Act 

School Improvement Program 

Improvement of Public 
School Support Program 

Vocat ional Education 



ESEA Title II Program 

Driver Education Program 

School Assistance in 
Federally Affected Areas 
Program 



Reporters 

Kenneth D. Streitmatter, Consultant, 
Special Programs Unit, Office of 
Continuing Education 

C. Phil Gore, Migrant Program Supervisor, 
Special Programs Unit, Office of 
Continuing Education 

Conrad Romero, Youth Community Relations 
Unit, Office of -Continuing Education 

Marvin J. Eakes, Improved Learning Unit, 
Office of Continuing Education 

Leonard Landry, Planning Unit Chief, 
Office of Planning and Evaluation 

Paul A. Bethke, Planning Unit, Office of 
Planning and Evaluation 

Leonard Hergenieter, Assistant Director 
for Program Planning and Development 

Paul Foster, Western Regional Planning 
Supervisor 

M. G. Hunt, Assistant Director for 
Professional Development 

Richard H. Edsall, Director of Evaluation 

James Wilson, Associate Director of the 
Division 

Roberta E. Young, Director of the 
Library Development and Programs 
Coordina t io . Uni t 

Robert Sundermeier, Improved Learning Unit, 
Office of Continuing Education 

John Holcomb, Consultant, Special Programs 
Unit, Office of Continuting Education 
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COLORADO (Cont.) 



Follow Through State 
Technical Assistance 
Program 

Compensatory Education 
Program 



ESEA Title III Grant 
Management Program 



Virginia Plunkett, Special Programs Unit, 
Office of Continuing Education 



Ward Vining, Director, Special Projects 
Unit, Office of Continuing Education 

Victor Wall, Special Projects Unit, 
Office of Continuing Education 

Byron Parks,- Special Projects Unit, 
Office of Continuing Education 

Roger Duncan, ESEA III Coordinator, 
Special Programs Unit, Office of Continuing 
Education 



Drug Abuse Program 



Special Education for 
Handicapped Children 
Program 



Transportation Grants 
Administration Program 



Adult Basic Education 
Program 

Testing Guidance and 
Counseling Program 

Intercultural Community 
Relations Program 

School Food Services 
Program 



Jan Schneider, Improved Learning Unit, 
Office of Continuing Education 

John Ogden, Director, Pupil Personnel Unit, 
Office of Continuing Education 

David Miles, Pupil Personnel Unit, 
Office of Continuing Education 

Tony Paulmeno, Pupil Personnel Unit, 
Office of Continuing Education 

Mary Ann Francis, Consultant for Transportation 
Grants Administration, Planning Unit, Office of 
Planning and Evaluation 

Joseph M. Connors, Improved Learning Unit, 
Office of Continuing Education 

Fred E. Holmes, Office of Continuing Education, 
Youth Community Relations Unit 

Earl W. Phillips, Youth Community Relations 
Unit of the Office of Continuing Education 

Daniel Wisotzkey of the Special Programs 
Unit of the Office of Continuing Education 
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